Decision Rule¶
See the backing repository for Skope Rules here.
Summary¶
Decision rules are logical expressions of the form IF ... THEN .... Interpret’s implementation uses a wrapped variant of skope-rules[2], which is a weighted combination of rules extracted from a tree ensemble using L1-regularized optimization over the weights. Rule systems, like single decision trees, can give interpretability at the cost of model performance. These discovered decision rules are often integrated into expert-driven rule-based systems.
How it Works¶
The creators of skope-rules have a lucid synopsis of what decision rules are here.
Christoph Molnar’s “Interpretable Machine Learning” e-book [1] has an excellent overview on decision rules that can be found here.
For implementation specific details, see the skope-rules GitHub repository here.
Code Example¶
The following code will train an skope-rules classifier for the breast cancer dataset. The visualizations provided will be for both global and local explanations.
from interpret import set_visualize_provider
from interpret.provider import InlineProvider
set_visualize_provider(InlineProvider())
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from interpret.glassbox import DecisionListClassifier
from interpret import show
seed = 1
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=seed)
dl = DecisionListClassifier(random_state=seed)
dl.fit(X_train, y_train)
dl_global = dl.explain_global()
show(dl_global)
dl_local = dl.explain_local(X_test[:5], y_test[:5])
show(dl_local)
Further Resources¶
Bibliography¶
- 1
Christoph Molnar. Interpretable machine learning. Lulu. com, 2020.
- 2
Florian Gardin, Ronan Gautier, Nicolas Goix, Bibi Ndiaye, and Jean-Mattieu Schertzer. Skope Rules: Machine Learning with Logical Rules in Python. 2017. URL: https://doi.org/10.5281/zenodo.4316671, doi:10.5281/zenodo.4316671.
API¶
DecisionListClassifier¶
-
class
interpret.glassbox.DecisionListClassifier(feature_names=None, feature_types=None, **kwargs)¶ Initializes class.
- Parameters
feature_names – List of feature names.
feature_types – List of feature types.
**kwargs – Kwargs passed to wrapped SkopeRules at initialization time.
-
explain_global(name=None)¶ Provides global explanation for model.
- Parameters
name – User-defined explanation name.
- Returns
An explanation object.
-
explain_local(X, y=None, name=None)¶ Provides local explanations for provided instances.
- Parameters
X – Numpy array for X to explain.
y – Numpy vector for y to explain.
name – User-defined explanation name.
- Returns
An explanation object.
-
fit(X, y)¶ Fits model to provided instances.
- Parameters
X – Numpy array for training instances.
y – Numpy array as training labels.
- Returns
Itself.
-
predict(X)¶ Predicts on provided instances.
- Parameters
X – Numpy array for instances.
- Returns
Predicted class label per instance.
-
predict_proba(X)¶ Provides probability estimates on provided instances.
- Parameters
X – Numpy array for instances.
- Returns
Probability estimate of instance for each class.
-
score(X, y, sample_weight=None)¶ Return the mean accuracy on the given test data and labels.
In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.
- Parameters
X (array-like of shape (n_samples, n_features)) – Test samples.
y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.
- Returns
score – Mean accuracy of
self.predict(X)wrt. y.- Return type
float